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Abstract 


Sufficiently developed skills in L1 are considered a precondition for success in the 
achievement of bilingualism. Moreover, it is reported that language learners whose 
native language skills are better mastered obtain superior second language profi- 
ciency and exhibit higher levels of language competence and accomplishment in L2. 
This chapter aims at investigating the effect of using test-takers’ mother tongue 
(Turkish as L1) and foreign language (English as L2) in reading comprehension 
questions in the language classroom. Two groups of adult participants preparing for 
the Language Exam for Higher Education Institutions (YOKDiL) were tested under 
two different conditions. The control group was exposed to the reading test with mul- 
tiple-choice questions and options in English, while the experimental group was pro- 
vided with questions and options in their L1. The results indicated that the experi- 
mental group participants scored significantly higher than the control group partici- 
pants, leading to several implications for the use of L1 in language assessment. 
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1. Introduction 


Research on the effect of L1 on success or ability in L2 has revealed that L1 
users activate their L1 knowledge in order to simplify the process of foreign 
language learning (Karim 2003). The already developed skills in L1 are con- 
sidered necessary for achieving bilingualism (Cummins 1979), and it is also 
indicated that learners whose native language skills are better mastered gain 
superior second language proficiency and exhibit higher levels of second lan- 
guage competence and second language classroom accomplishment (Sparks 
et al. 2008; Artieda 2017). 


In addition to the effects of L1 skills on L2, first language use in language 
classrooms has been debated for many decades and various issues have 
been voiced on using or not using learners’ L1 in the classroom around the 
world (Cook, Singleton 2014; Lee 2018). The debates include arguments for 
the negative and positive roles of L1 in L2 learning contexts (Du 2016; Macaro 
et al. 2018). These debates are often on whether to avoid explanations or 
translations in L1 in specifically grammatical and oral tasks (Jin, Cortazzi 
2018), and on what roles the use of L1 might have in language classrooms 
(Krajka 2004; Sali 2014) and in tests when investigating cross-cultural per- 
spectives such as L1 use in different contexts and tasks such as translation 
(Krajka 2019). It is reported that language learners who better master their na- 
tive language skills obtain superior second language proficiency and exhibit 
higher levels of language competence and accomplishment in L2 (Sparks et 
al. 2008; Artieda 2017). 


Although there are various differences regarding linguistic knowledge at the 
initial stages (Table 1), L2 learners are believed to transfer their reading skills 
in L1 to L2. Therefore, learners with limited literacy abilities in their L1 are not 
expected to use or transfer strategies or abilities such as problem-solving abili- 
ties to L2 reading processes (van Gelderen et al. 2004). This is supported by 
the developmental interdependence hypothesis, stating that L2 reading devel- 
opment is greatly supported by L1 literacy (Cummins 1991). 


L2 learners are then at an advantage as they can benefit from pre-existing 
and tested reading skills in L1, although their linguistic knowledge in L2 may 
not be developed and readily available. The question to ask is, then, whether 
Li use in language assessment, specifically in reading comprehension tests, 
can also benefit language learners. 
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Table 1. Linguistic knowledge at initial stages of L1 and L2 reading based on Grabe and 
Jiang (2018). 


L1 reading L2 reading 
e Learn to read after speaking e Reading and speaking at the same time 
e Master speaking before reading e Master reading before speaking 
e Grammatical structures as implicit e Grammatical structures as explicit 
knowledge knowledge 
e Already learned structures / devel- e Linguistic knowledge not developed yet 
oped linguistic knowledge but pre-existing reading skills in L1 


2. Literature review 


Research conducted on L1 use as the question language in reading as- 
sessment has investigated this issue in various perspectives. Previous re- 
search has shown that L1 strategies were used by L2 learners in different con- 
texts with various advantages. For example, Nevo (1989) compared the strat- 
egy use of forty-two tenth-grade students of French in a reading comprehen- 
sion test with questions in a multiple-choice format with four alternative an- 
swers, where the question language was French and Hebrew. One of the find- 
ings indicated that, when the participants had high linguistic knowledge and 
proficiency in their L1, they were more likely to benefit from the strategies that 
would help them find the correct answer. Another study, conducted by Seng 
and Hashim (2006), investigated the use of L1 reading strategies utilized and 
the possible reasons for using them in L2 reading classes. The participants 
included randomly selected four female freshmen in an Education course at 
Universiti Pendididkan Sultan Idris, who were asked to think aloud while read- 
ing L2 texts in a group. Based on the analysis of the protocols, it was deter- 
mined that L1 strategies were used by all participants due to word- and idea- 
related difficulties they encountered in L2. 


The few existing studies investigating the effect of question language on test 
scores have primarily focused on students at the beginning levels of language 
learning (Hock, Poh 1979; Shohamy 1984; Godev et al. 2002) largely ignoring 
advanced-level readers (except Cox et al. 2019) for whom the effect of the 
question language may be less significant. Furthermore, the research that has 
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been conducted has focused on commonly-taught L2s such as English, 
French, and Spanish. 


Hock and Poh (1979) investigated the effects of presenting passages in the 
pre- and post-tests where passages were provided in English and the delayed 
post-test where the multiple-choice questions were in Bahasa Malaysian. The 
participants were 39 students in a one-month intensive English course. The 
findings indicated that when the questions were presented in the participants’ 
L1, their performance improved significantly, which might be attributed to the 
fact that the participants could understand the questions and options when 
provided in L1; the participants with weaker abilities in English might have 
benefited from that. Similarly, investigating the effect of the testing method, 
Shohamy (1984) studied multiple-choice (MC) and open-ended (OE) questions 
presented in L1 (Hebrew) and L2 (English) texts. The study included control 
and experimental phases and was conducted twice with different participants 
from the population. In the first experiment, the control part included eight 
short texts with 8 MC English questions, while participants, in the experimental 
part, were exposed to ‘Books’ test with 8 MC English, 8 MC Hebrew, 8 OE 
English, and 8 OE Hebrew questions. The second experiment included the 
same procedure, but a longer text in the experimental part, the ‘Jerusalem’ 
text. The findings indicated that MC questions were found easier than the OE 
procedure in two different texts and that providing questions in learners’ L1 
reduced the difficulty level of the reading comprehension. 


Godev et al. (2002), on the other hand, analyzed the effects of using learn- 
ers’ L1 and L2 in open-ended questions for reading comprehension. The par- 
ticipants included college-level learners of Spanish and were exposed to dif- 
ferent conditions including the combinations of L1 and L2 use in a reading test 
with seven open-ended questions. The results indicated that the learners’ per- 
formance increased when they were presented questions in L1. 


A very recent study conducted by Cox et al. (2019) aimed at investigating 
the effects of question language (in L1 and L2) on advanced learners of Rus- 
sian and their attitudes. The participants included 64 advanced-level readers 
enrolled in a third-year Russian class with experience in Russian-speaking 
countries. The participants responded to a test of two ten-question parts with 
20 MC questions in total in their native language, English, and their second 
language, Russian. The results revealed that the participants scored higher on 
the test whose question language was their native language. The results also 
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indicated that the participants who answered questions in their native lan- 
guage, English, had a low anxiety level and a higher level of confidence. 


The studies reviewed above indicate several positive effects of using L1 as 
the question language in reading comprehension questions in limited item for- 
mats and that mainly learners with limited proficiency in L2 benefit from this. 
However, the findings are inconclusive considering that there are few studies 
conducted on the issue of question language and almost all fail to consider 
learners’ views and suggestions on this. The present study aims to fill these 
gaps by investigating the effects of question language on intermediate learners 
of English preparing for a language proficiency test in Turkey. This chapter, 
therefore, aims at investigating the effect of using the test-takers’ mother 
tongue (Turkish as L1) and foreign language (English as L2) in the test ques- 
tions for reading in the language classroom and also consider the participants’ 
views and suggestions. 


3. Methodology 


In line with this aim of the study, the following research questions were pro- 
posed to investigate the effect of question language on the participants’ 
scores: 


1. Is there a statistically significant difference in the reading comprehen- 
sion scores of control and experimental group participants who were 
exposed to questions in English and in Turkish? 


2. What are the views of the learners of English towards answering the 
questions in their L1 (Turkish)? 


3.1. Research design 


The study benefited from a mixed-method research design including instru- 
ments to collect quantitative and qualitative data. The quantitative data were 
used to determine any significant difference between the control and experi- 
mental groups. The qualitative data, on the other hand, were collected to find 
out the views of the participants regarding answering the questions in L1. 
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3.2. Participants 


The participants of the study were general practitioners (n=29) who were 
preparing for the Language Exams for Higher Education Institutions (YOKDIL), 
which was required to qualify for specialization examination and education in 
Turkey (Turan, Uner 2015). All the participants had had at least eight years of 
English instruction throughout their education, and their ages ranged from 27 
to 38. Based on the proficiency test given by the private language institution 
which the participants attended, their level of English was determined to be 
intermediate. Moreover, all the participants shared Turkish as their L1. 


3.3. Data collection and procedure 


Two groups of adult participants (n=29) preparing for Language Exam for 
Higher Education Institutions (YOKDIL / YDS) were tested under two different 
conditions. The control group included 14 general practitioners and was ex- 
posed to the reading test which included multiple-choice questions in English, 
while the experimental group included 15 general practitioners and was pro- 
vided with questions and options in their L1. The reading test included four 
texts, each of which included five multiple-choice questions with five options, 
and the exam duration was 35 minutes (See the appendix for an example 
test). 


During the second week of the training, the participants were contacted 
through the institution and were informed briefly regarding the study without 
going into details. Upon their consent, the participants were divided into two 
groups by the private language institution based on the proficiency test at the 
very beginning of their training. Therefore, there was no need for a pre-test to 
indicate that the participants had the same proficiency level and no specific 
treatment or instruction by the researcher, as the institution followed the cur- 
riculum based on the language exam that the participants were required to 
take. At the end of the third week, the participants were asked to answer twen- 
ty reading comprehension questions in four medical texts. The questions were 
of multiple-choice format and included five options. The control group was ex- 
posed to all the questions and options in English, while the experimental group 
participants were exposed to the same four texts, but the questions and op- 
tions were presented in their L1, Turkish. One week after the treatment, semi- 
structured interviews were held with the participants in the experimental group 
regarding their views on the use of L1 as the question language. 
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3.4. Data analysis 


In order to determine whether there would be statistically significant differ- 
ences between the two groups, the post-test data based on the responses to 
the reading questions were analyzed using the nonparametric Independent 
Samples Mann-Whitney U test. An independent Samples t-test was not used, 
as the population distribution was not normal and was heavily skewed. The 
responses obtained through semi-structured interviews were subject to content 
analysis. The transcriptions of the interviews were analyzed for emerging 
themes and codes by the researcher and another expert in English language 
teaching. 


4. Findings and discussion 


Research Question 1: Is there a statistically significant difference in the 
reading comprehension scores of control and experimental group participants 
who were exposed to questions in English and in Turkish? 


In order to find an answer to the first research question, an Independent 
Samples Mann-Whitney U test was calculated. The results indicated that the 
experimental group participants scored higher than the control group partici- 
pants, leading to several implications regarding the use of L1 in language as- 
sessment (Table 2). 


Table 2. The statistical analysis conducted on the posttest. 


Group Mean Std. Deviation Median 
Control 11.57 1.505 12.00 
Experimental 14.53 2.167 15.00 
Total 13.10 2.381 13.00 


Table 2 indicates that the experimental group participants had a mean score 
of 14.53, which is higher than the control group participants with a mean score 
of 11.57. 
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The test results also revealed a statistically significant difference in the 
scores of the experimental group participants (Md = 15, n = 15) and of the con- 
trol group participants (Vd = 12, n = 14), U= 182, z = 3.429, p= .001, r= .6 
(Tables 3 and 4). 


Table 3. Rank results. 


Group N Mean Rank Sum of Ranks 

Post-test Control 14 9.46 132.50 
Experimental 15 20.17 302.50 
Total 29 


Table 4 below provides a quick summary of the results of the test. In the 
analysis, the Sig. value is .001. This is less than .05 and therefore, the differ- 
ence between these two groups is significant and favors the participants in the 
experimental group, who were exposed to Turkish as the question language in 
the reading comprehension test. Moreover, Cohen’s (1988) statistical power 
analysis was also conducted in order to determine the effect size statistic for 
the Mann-Whitney U test and it was determined as .6, which proved to be a 
large effect. 


Table 4. Independent-samples Mann-Whitney U test results. 


Statistic Result 
Total N 29 
Mann-Whitney U 182.500 
Wilcoxon W 302.500 
Test Statistic 182.500 
Standard Error 22.603 
Standardized Test Statistic 3.429 
Asymptotic Sig. (2-sided test) 0.001 
Exact Sig. (2-sided test) 0.000 
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This finding is consistent with that of the studies conducted by Hock, Poh 
(1979), Shohamy (1984), and Godev et al. (2002) in that the use of L1 as the 
question language resulted in a significant increase in the scores of the partic- 
ipants. It is also necessary to note that these studies included participants with 
limited proficiency in L2. However, the participants of the current study were 
intermediate learners of English and in the same vein, their scores increased. 
Therefore, based on this finding, it can be stated that the positive effect of us- 
ing L1 as the question language cannot be limited to the participants at low 
levels or with limited proficiency. 


Research Question 2: What are the views of the learners of English towards 
answering the questions in their L1 (Turkish)? 


The analysis of these responses has led to the themes with categories and 
corresponding codes as indicated in Table 5. In order to determine the views 
of the participants regarding the use of L1 (Turkish) as the question language, 
the most representative quotes were selected. Considering all the responses, 
it can be stated that the participants had an overall positive experience with 
answering questions in L1 and the majority indicated preference for L1 ques- 
tions. The responses resulted in two themes, four categories, and seven 
codes, which have been briefly discussed below. 


Table 5. The theme, categories, and corresponding codes. 


Theme Categories Codes 
Question difficulty Language structures 
Benefits of L1 as the ques- Wording 


tion language 


Psychological effects Positive attitudes 


Low anxiety 


Suggestions on the question Combining L1 andL2 __ Use of both languages 
language 


Different item formats | Open-ended questions 


Short answer 
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Regarding the benefits of using L1 as the question language, the majority of 
the participants (n=25) stated two major merits: question difficulty and psycho- 
logical effects. According to the participants, as the questions and the options 
in the multiple-choice questions were provided in their L1, it was easy to un- 
derstand what was requested due to their familiarity with the language struc- 
tures. In other words, the participants stated that in L2, which was English in 
this study, it could be difficult for them to understand what was said or implied 
in the options although they stressed that they understood the whole passage. 
When the questions in the previous exams were checked, it was noticed that 
the options in the multiple-choice questions included either synonyms and/or 
antonyms of the words used in the reading passage. This was most probably 
done to avoid revealing the answer by using different words and/or structures 
so that learners could not find the same words in the reading test. In other 
words, in order not to use same words and therefore not to lead the test takers 
to the correct option, either different structures (passive voice instead of active 
voice) or a synonym or a different conjunction with the same purpose (use of 
‘due to’ rather than ‘because of for example) were used. In such cases, learn- 
ers may not understand the questions due to unknown vocabulary in the ques- 
tions and options and the complex structures, resulting in a failure in selecting 
the correct option, although they might have comprehended the text. This find- 
ing is consistent with that of the study by Seng and Hashim (2006), which indi- 
cated that the participants benefited from L1 strategies mainly due to wording 
in L2. This was clearly stressed by one of the participants as follows: 


| can say that | have understood every single line of the text. | can 
summarize or even talk about the details. However, when it comes to 
the questions, it is rather difficult. The questions include different 
structures and words rather than the ones in the text and this makes 
answering the questions difficult although you could understand what 
was communicated in the text. 


The participants also voiced positive attitudes and low anxiety, which were 
categorized under ‘psychological effects’. Most of the participants (n=22) ex- 
pressed an opinion that answering questions in their L1 was perceived posi- 
tively and added that this increased their self-confidence. In addition, a great 
majority of the participants stated that they had lower anxiety than when an- 
swering questions in English, which corroborates the finding of the study by 
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Cox et al. (2019), which reported a low anxiety level among the participants. 
This might be attributed to the fact that the participants, as they had to deal 
with short texts, felt less anxious as the questions and options were provided 
in L1. This is clearly stated in the following quotation: 


| really liked answering questions in L1. | think there are two reasons 
for this. One is that, as | knew that questions would be in Turkish, | 
thought that | could answer the questions easily without suffering 
from anxiety. | was really anxious about reading comprehension 
questions, but the questions in Turkish decreased my negative feel- 
ings towards the questions. 


The participants were also asked to share any suggestions regarding the 
question language in reading comprehension tests. Several participants (n=15) 
voiced the need to combine L1 and L2 in the tests and suggested using both 
languages in the questions. In other words, it was suggested that some ques- 
tions should be provided in L1 and some in L2. They stated that they were not 
completely against using L2 questions, as it was also important to be exposed 
to those questions. One participant underscored this as follows: 


| think there must be questions in learners’ L1 and L2. By asking 
questions in L1, learners can be encouraged to benefit from their L1. 
However, | do not think that questions should be completely in Turk- 
ish. They must be in English, too. We must also be asked questions 
in English and be exposed to it. 


As for the other category, different item formats, they included open-ended 
questions and short-answers. Several participants (n=17) stated that L1 could 
also be used in open-ended questions and short answers and added that 
reading texts could be accompanied by questions in L1 and L2 and learners 
could be asked to provide their answers in these languages. One of the partic- 
ipants expressed her views as follows: 


We should also try using Turkish in other question types in addition 
to multiple-choice questions. | mean we can be asked open-ended 
questions in L1 and/or L2 and we can answer these questions in our 
language or English. It would be interesting to do it. 
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Considering all the responses of the participants, it can be stated that the 
use of L1 is appreciated by the participants as it seems to have decreased 
their anxiety level but increased their self-confidence. Moreover, as the partici- 
pants clearly expressed, although they had preference for L1 use in the ques- 
tions, L2 must also be used as the question language. 


5. Pedagogical implications 


Based on the findings obtained in the study, the following implications can 
be put forward as for the use of L1 and L2 as the question language in the 
reading comprehension tests: 


e The aim of a reading comprehension test is to test learners’ ability to 
understand L2 texts. Therefore, learners can be provided with questions 
in L1 or they can be allowed to respond in L1 to determine whether they 
can also prove that they have understood the text. 


e Certain learners, depending on their own needs, may benefit significant- 
ly from reading in L2 and responding in L1 or vice versa. For example, 
people working in the field of medicine need reading ability and bilingual 
fluency more than those from other fields as they need to be updated 
with the new developments and improvements in the field. 


e L2 as the question language cannot be avoided as learners must also 
be exposed to the target language as much as possible. However, 
learners’ L1 can also be used in the questions or in the answers. 


e Li and L2 can be used in different item formats such as open-ended 
and short-answer questions. Moreover, the task of summarizing in L1 
and L2 can also be utilized. 


6. Conclusions 


The current study aimed to investigate the effect of using L1, Turkish as the 
question language in the L2, English reading short texts and to determine the 
participants’ views and suggestions. The participants included 29 general 
practitioners preparing for language proficiency tests in Turkey. The findings 
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indicated that the experimental group participants scored higher in the reading 
comprehension questions in Turkish than those in the control group, suggest- 
ing various implications. Using L1 as the question language seems to have a 
positive impact on learners’ number of correct answers, anxiety and confi- 
dence levels. The participants prefer L1 questions, but are not against L2 
questions; however, using L1 as the question language seems to have allowed 
the participants to show their ability to understand text through L1, as they 
were not overburdened by the change in the language structures or use of 
synonyms in the questions and options. 


7. Limitations and further research directions 


The study suffers from various limitations. The study recruited a limited 
number of participants for the study due to several reasons. Moreover, it was 
not possible to control variables such as the effects of practice or training out- 
side the classroom. Regarding methodology, several improvements can also 
be made. The study included a relatively short treatment period with only one 
post-test and short texts of reading with multiple-choice questions. Therefore, 
further research can conduct studies with longer treatment periods together 
with post-tests and delayed post-tests to investigate the effect of retention as 
well. Finally, further research can also utilize different tasks while investigating 
the effects of using L1 as the question language, such as open-ended ques- 
tions and summarizing. 
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